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3  INTRODUCTION 

The  US  Army  and  society  in  general  is  rapidly  developing  towards  greater  dependence  on  fragile 
technological  and  management  systems  vulnerable  to  natural  disasters  and  terror  assaults. 

Economic  and  sociologic  global  polarization  has  and  will  continue  to  produce  an  unstable  and 
insecure  international  political  situation  where  conflicts  and  confrontations  are  numerous. 
Conflicts  that  occur  will  seldom  be  contained  in  one  area  and  could  erupt  anywhere. 

The  type  of  society  we  prefer  is  open  and  liberal,  thus  traditional  countermeasures  against 
terrorism  will  be  ineffective.  Possibly  the  most  effective  response  to  any  future  crisis  scenario 
will  be  the  ability  to  manage  and  adapt  to  the  situation.  On  a  infrastructural  level,  as  well  as,  on 
the  individual  level. 

To  develop  these  management  skills,  training  and  testing  is  essential.  However,  full-scale 
training  and  practice  events  are  very  expensive,  logistically  complicated  and  therefore  cannot  be 
conducted  frequently. 

There  is  an  escalating  need  in  the  US  Army  to  simulate  crisis  situations.  These  simulations  can 
be  used  to  calculate  the  effects  of  different  actions  and  countermeasures.  Most  importantly,  these 
simulations  can  be  used  to  train  US  Army  management  and  emergency  personnel. 

Mobile  Media  Inventions  AS  has  over  the  past  decade,  developed  a  set  of  technologies  that  when 
merged  together  can  produce  the  framework  for  this  type  of  simulation  system.  These 
technologies  have  evolved  from  medical  and  entertainment  3D  animation,  mobile  telecom 
interactive  technology  and  satellite  and  aerial  photography.  When  merged,  developed,  backed  up 
by  sufficient  statistical  data  and  installed  in  a  main  frame  computer  system,  a  structural  and 
sociological  model  can  be  generated. 

The  primary  objective  of  this  project  was  to  develop  a  complete  multi-level  simulation  system 
for  disaster  events  including  terrorist’s  attacks.  The  first  phase  of  development  produced  a 
working  theoretical  model  to  merge  the  different  technologies  in  small  scale.  Results  of  this 
developmental  phase  will  then  help  to  create  the  foundation  for  building  a  full  scale  simulator. 

4  BODY 

4,1  Significance  and/or  Uniqueness  of  the  Effort: 

To  develop  an  interactive  emergency  simulation  and  response  system,  solutions  to  several 
fundamental  problems  outlined  below  must  be  found.  Mobile  Inventions  has  been  addressing 
these  problems  with  a  strong  focus  and  significant  commitment  for  over  a  decade  (company 
established  in  1993).  During  these  years,  the  results  of  basic  research  in  mathematics,  physics 
and  computer  graphics  from  Europe  and  the  US  have  been  utilized. 
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Mobile  Inventions  and  its  subsidiaries  have  patents  and  patents  pending  that  address  solutions  for 
surgical  interfacing,  suturing,  soft  tissue  modeling  (Sim3DM)  and  methods  for  representing 
geometric  shapes  and  geometric  structures  in  computer  graphics.  Within  the  corporate  structure 
of  Mobile  Media  is  world  class  expertise  in  3D  animation  and  graphical  entertainment  solutions, 
mobile  communication  and  content  handling  and  3D  animation.  Mobile  Inventions  has  access  to 
cutting-edge  technology  and  competence  within  the  field  of  computer  graphics  and  computer 
networks  for  distributed  solutions.  The  company  with  Ericsson  developed  and  demonstrated  the 
worlds  first  mobile  interactive  news  application  at  Telecom  1999  utilizing  2Do3D™  3D 
animation  software.  In  2000  several  development  projects  were  launched  together  with  Intel. 
The  Mobile  Inventions  subsidiary,  SimSurgery,  has  roots  partly  from  the  Interventional  Centre  at 
the  Norwegian  National  Hospital.  This  research  centre  for  telemedicine  and  surgical  robotics  was 
one  of  the  first  European  sites  performing  robot  assisted  operations. 

The  4  fundamental  problems: 

1 )  Automatic  generation  of  scenery  models  from  satellite  images  and  photos 

Over  the  past  decade,  Mobile  Media  Inventions  has  cooperated  with  the  leading  Norwegian 
research  institutes  (NR  and  Sintef)  to  find  a  solution  to  this  problem.  In  the  IMST  project, 
approximately  $3.0M  was  invested  towards  the  design  and  implementation  of  a  system  for 
building  an  urban  3D  structural  model  based  on  a  combination  of  satellite  images  and  photos. 

The  project  aims  to  find  solutions  for  a  set  of  issues  related  to  geographical  database  technology, 
automatic  interpretation  of  high-resolution  satellite  images  and  stereo  models  and  representation 
and  rendering  on  different  devices. 

The  IMST  concept  comprises  extensive  use  of  high-resolution  optical  imagery.  Generation  of 
stereo  models  consists  of  determining  the  acquisition  geometry  of  images  that  will  be  needed  to 
retrieve  the  3D  positions  of  details  of  the  ground  surface  and  its  objects.  In  order  to  generate  the 
stereo  model,  a  model  is  needed  for  image  acquisition  geometry  and  we  need  to  determine  the 
parameters  of  the  model.  These  are  determined  by  means  of  corresponding  tie  points  between  the 
images  and  ground  control  points,  which  refer  to  a  global  co-ordinate  system. 

The  tie  point  generation  has  reached  a  high  degree  of  automation  but  the  methods  need  to  be 
tested  and  tuned.  In  general,  automated  methods  are  expected  to  yield  higher  quality  tie  points 
than  manual  methods.  It  is  uncertain  whether  these  methods  will  work  for  satellite  data  over 
urban  areas.  The  automatic  identification  of  ground  control  points  is  a  more  difficult  task  which 
is  not  completely  solved.  The  success  of  this  task  will  depend  on  how  the  images  can  be 
interpreted  and  understood  in  terms  of  existing  available  data.  A  strategy  is  proposed  where 
existing  data  are  rendered  in  a  manner  that  makes  them  similar  to  the  image  data.  When  existing 
data  are  poor,  manual  intervention  must  be  expected. 

2)  3D  visual  content  and  simulation  production  software. 

AnimagicNet  was  founded  in  1995  by  Mobile  Media  Inventions  as  a  subsidiary.  ANET  is  now  a 
leading  digital  animation  studio  with  creative,  technical  and  production  capabilities  to  deliver  a 
new  generation  of  3D  animated  feature  films  and  related  media  products.  Its  activities  as  an 
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animation  studio  are  based  in  the  same  technological  development  effort  that  has  produced  the 
foundation  for  SimSurgery  AS. 

As  a  studio  the  company  utilizes  off  the  shelf  animation  and  motion  capture  software,  but  adapts 
this  to  special  requirements  in  each  and  every  case.  This  capability  and  knowledge  has 
positioned  the  company,  from  an  animation  point  of  view,  at  the  same  level  as  much  larger 
organizations  like  for  example  Hollywood  based  PIXAR.  Based  in  this  knowledge  and 
connected  with  3D  gaming  graphics,  the  above  software  will  be  developed. 

3)  Realistic  and  real-time  medical  3D  simulation  solutions. 

The  company  SimSurgery  has  been  addressing  these  problems  with  a  strong  focus  and 
significant  commitment  for  five  years  (company  established  by  Mobile  Inventions  in  1999). 
During  these  years,  results  of  basic  research  in  mathematics,  physics  and  computer  graphics  from 
Europe  and  the  US  has  been  utilized.  SimSurgery  has  patents  pending  that  address  solutions  for 
surgical  interfacing,  suturing  and  soft  tissue  modeling.  (Sim3DM) 

SimSurgery  is  part  of  a  corporate  structure  with  expertise  in  3D  animation  and  graphical 
entertainment  solutions.  SimSurgery,  together  with  AnimagicNet,  has  access  to  cutting-edge 
technology  and  competence  within  the  field  of  computer  graphics  and  computer  networks  for 
distributed  solutions. 

4)  Fast  and  reliable  interaction  with  mobile  devices  to  exchange  and  render  infonnation 

Finding  solutions  to  this  problem  has  been  the  core  activity  of  Mobile  Media  Inventions  since  the 
mid  90 ’s.  In  1999,  Mobile  Inventions  developed  the  world’s  first  multimedia  solution  for  mobile 
devices  for  Ericsson.  The  system  was  successfully  demonstrated  at  the  Telecom’99  Expo.  Since 
then,  the  solution  has  been  improved  and  made  available  for  other  handsets  and  platforms.  Our 
goal  has  been  to  be  able  to  scale  3D  animation  stream  and  present  it  with  the  right  resolution 
according  to  device  characteristics.  The  aspect  of  geographical  3D  model  data  transmission  and 
rendering  has  been  addressed  in  the  above  mentioned  IMST  project.  The  introduction  of  Java 
and  picture  phones  to  the  marketplace  has  opened  totally  new  possibilities  within  this  field. 
Utilizing  these  technologies  makes  it  possible  to  integrate  mobile  devices/users  by  collecting 
pictures  from  the  user  and  sending  pictures/maps  and  other  information  to  them.  It  is  also 
possible  to  distribute  applications  to  be  used  for  a  particular  event,  in  real-time.  These 
applications  can  be  tailored  to  guide  the  user  in  an  emergency  situation  and/or  retrieve 
fundamental  multimedia  information  from  the  scene  and  relayed  back  to  the  emergency  center. 
In  this  manner,  real  situation  data  can  be  transferred  and  merged  into  the  3D  simulation  model. 

4.2  Hypothesis 

1)  It  is  feasible  to  automatically  generate  3D  scenery  models  obtained  from  remote  cameras 
in  satellites,  helicopters,  etc. 

2)  An  anatomy  explorer  derived  from  medical  images  facilitates  training  surgical 
intervention. 
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4.3  Technical  Objective 


The  primary  objective  of  the  proposed  project  is  to  develop  a  complete  multi-level  simulation 
and  response  system  for  disaster  events  including  terrorist’s  attacks. 

4.4  Military  Significance: 

There  is  an  escalating  need  in  the  US  Army  to  simulate  crisis  situations  such  as  terrorist’s 
attacks.  These  simulations  can  be  used  to  calculate  the  effects  of  different  actions  and 
countenneasures.  Most  importantly,  these  simulations  can  be  used  to  train  US  Anny 
management  and  emergency  personnel.  The  development  of  a  realistic,  interactive  emergency 
simulation  and  response  system  is  an  integral  component  to  military  readiness  and  preparedness 
to  terrorism. 

From  a  US  Army  stand  point,  defensive  rescue  scenarios  and  situations  could  be  simulated 
parallel  to  offensive  interventions  and  operations.  The  simulator  will  also  allow  for  very  specific 
urban  warfare  scenarios.  These  simulations  would  create  possibilities  to  test  different  tactical 
options  in  a  given  situation  from  an  offensive,  as  well  as,  defensive  point  of  view.  Realism  is  a 
key  word  in  connection  with  the  concept. 

4.5  Methods 

The  Middletown  concept  comprises  extensive  use  of  high-resolution  optical  imagery.  The 
generation  of  stereo  models  involves  the  acquisition  geometry  of  the  images  needed  to  retrieve 
the  3D  positions  of  ground  surface  details.  In  order  to  generate  the  stereo  model,  a  model  for 
image  acquisition  geometry  is  needed  and  we  need  to  determine  the  parameters  of  the  model. 
This  is  determined  by  means  of  creating  corresponding  tie  points  between  the  images  and  ground 
control  points  that  refer  to  a  global  co-ordinate  system. 

Tie  point  generation  has  reached  a  high  degree  of  automation  but  the  methods  need  to  be  tested 
and  tuned.  In  general,  automated  methods  are  expected  to  yield  a  higher  quality  of  the  tie  points 
than  manual  methods.  It  is  uncertain  how  they  will  work  for  satellite  data  over  urban  areas. 

Automatic  identification  of  ground  control  points  is  a  more  difficult  task  that  is  not  completely 
solved.  The  success  of  this  task  depends  on  how  the  images  can  be  interpreted  and  understood  in 
tenn  of  existing  available  data.  A  strategy  is  proposed  that  the  existing  data  are  rendered  in  a  way 
that  makes  them  similar  to  the  image  data.  Further  investigation  is  required  to  test  experimentally 
how  current  available  methods  actually  work  for  project  images. 

The  generation  of  digital  terrain  models  from  stereo  imagery  has  become  a  regular  procedure  in 
mapping  the  ground  surface.  Commercially  available  automatic  systems  perform  reasonably  well 
but  rely  to  some  extent  on  successful  matching  of  the  area  in  question.  Automatic  object 
extraction  from  multi-view  images  in  urban  areas  is  today  a  unsolved  task,  at  least  for  practical 
applications.  However,  rapid  progress  has  been  made  and  there  is  active  research  going  on. 

Difficulties  come  from  not  only  the  complexity  of  the  urban  scene  itself  but  also  from  the  weak 
knowledge  about  the  principle  of  human  visual  perception.  Also,  low  success  rates  are  due  to 
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severity  in  structuring  and  semantic  attribution.  Other  problems  involve  the  lack  of  an  accepted 
model  for  the  object  "building"  and  the  diversity  of  applications  for  3D  building  data. 

So  far,  reported  successes  are  for  very  special  conditions.  Only  small  areas  are  possible  at 
reasonable  processing  time.  Multiple  steps  are  necessary  to  detect,  verify  and  reconstruct  urban 
structures  from  satellite  imagery.  Many  types  of  geographic  data  have  to  be  represented  in  the 
system.  We  need  to  study  solutions  for  import,  storing,  updating  and  presentation  of  geographic 
data.  Existing  GIS  and  database  management  systems  together  with  visualisation  packages  also 
need  to  be  investigated. 

For  GIS  and  regular  database  functionality  there  exist  commercial  solutions.  There  also  exist 
commercial  visualization  packages  that  can  present  rendered  terrain  modelswith  3D  CAD  model 
objects  included.  However,  there  are  doubts  whether  these  tools  can  satisfy  our  efficiency 
demands.  An  effective  treatment  of  terrain  models  for  large  areas  at  a  high  speed  is  required. 

There  is  ongoing  research  on  techniques  for  improving  the  efficiency  of  this  type  of 
visualization.  There  are  research  entities  in  Norway  pursuing  work  on  visualization  and  the 
project  might  benefit  from  this  competence.  A  major  focus  of  this  project  will  be  to  identify  and 
study  results  from  various  research  projects. 

In  building  an  anatomy  navigator,  the  project  seeks  to  implement  solutions  for  importing  medical 
3D  images  and  segment  anatomical  structures.  Modern  graphic  cards  used  in  games  will  be  used 
to  visualize  the  anatomy  and  surgical  instrument  simulation  will  be  performed  by  using  surgical 
simulator  interfaces. 

The  use  of  mobile  devices  as  a  user  interface  to  the  simulator  will  set  high  constraints  on  both  the 
user  interface  and  the  communication  between  the  mobile  device  and  servers.  The  project  will 
study  new  technologies  and  concepts  regarding  this  issue  and  recommend  solutions. 

5.  KEY  RESEARCH  ACCOMPLISHENTS 

The  project  was  divided  into  several  Work  Packages  (WP).  The  following  WPs  are  defined: 

1)  Project  Management 

2)  Automatic  generation  of  scenery  models  from  satellite  images  and  photos 

3)  3D  visual  content  and  simulation  production  software. 

4)  Realistic  and  real-time  medical  3D  simulation  solutions. 

5)  Fast  and  reliable  interaction  with  mobile  devices  to  exchange  and  render  information 

6)  Requirement  specification 

7)  Phase  II  Planning  and  Budget 


5 


Project  Management: 

Goals 

This  WP  includes:  overall  project  management,  status-,  and  planning  meetings,  resource 
allocation  and  reporting.  The  main  goal  was  to  track  and  verify  the  progress  in  the  other  WPs  to 
ensure  that  the  results  are  delivered  within  the  defined  time,  quality  and  budget.  The  work  is  lead 
by  the  Project  Director  assisted  by  administrative  project  secretary. 

Results 

Due  to  resource  challenges,  especially  during  Q2,  the  project  was  granted  a  3  month  extension. 
The  end  date  was  changed  from  September  2006  to  December  2006. 

WP  3  (3D  visual  content  and  simulation  production  software)  has  not  been  prioritized  during  this 
phase.  The  main  reason  is  the  rapid  development  of  such  software,  especially  for  games.  We  also 
need  to  integrate  with  existing  solutions  used  in  military  systems.  The  work  in  this  phase  has 
been  shifted  towards  development  of  mobile  solutions  for  geographical  information  services  and 
location  services  as  the  market  already  are  in  a  need  of  such  systems  and.  As  Mobile  Media 
Invention  has  unique  knowledge  and  experience  in  such  technology,  it  has  been  natural  to 
reschedule  these  activities.  This  has  been  done  during  the  last  phase  of  this  project  (Q4  and  Q5). 

Automatic  generation  of  scenery  models  from  satellite  images  and  photos: 

Goals 

Mobile  Media  has  for  several  years  cooperated  with  the  leading  Norwegian  research  institutes 
NR  and  Sintef  to  find  a  solution  for  this  problem.  In  the  IMST  project,  close  to  USD  3.0  million 
has  already  been  invested  in  designing  and  implementing  a  system  for  building  an  urban  3D 
structural  model  based  on  a  combination  of  satellite  images  and  photos.  The  goal  of  this  WP  was 
to  integrate  the  work  and  results  from  the  IMST  project  with  the  Middletown  project.  The  WP 
should  also  investigate  other  related  technologies  and  methods  to  generate  necessary  scenery. 

Results 

There  has  been  produced  a  large  amount  of  applications  for  geographic  information  systems. 
Google  Earth,  Microsoft  Virtual  earth  has  introduced  advanced  aerial  imagery  services  during 
the  phase  I  of  this  project.  Google,  Microsoft  and  Yahoo  already  provides  various  API’s  which 
enables  light  integration  of  maps  and  other  information  in  applications  and  web  services.  This  is 
a  clear  indication  that  there  will  be  offered  new  technology,  services  and  standards  that  will 
support  Middletown  needs. 

Important  technological  solutions  for  the  Middletown  2010  project  have  been  investigated  in 
detail  in  the  IMST  project  carried  out  by  the  Norwegian  Computing  Center  (NR)  headed  by  the 
research  group  for  Statistical  Analysis,  Image  Analysis  and  Pattern  Recognition  (SAMBA)  in 
order  to  understand  exactly  what  Middletown  can  use  from  available  state-of-the-art  solutions 
and  what  has  to  be  developed  by  the  project.  The  following  topics  were  studied:  image 
orientation,  image  interpretation,  image  rendering,  land-surface  object  representation,  GPS 
positioning  accuracy,  and  systems  related  to  Middletown. 
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3D  city  models  play  an  important  role  in  Middletown  2010.  The  status  on  development  of  such 
3D  models  is  currently  not  so  advanced  as  on  regular  GIS  data.  Middletown  needs  more  work 
done  within  this  field  to  be  able  to  reach  the  goal  of  mapping  city  3D  model  (house  etc)  with 
terrain  models.  The  construction  of  a  3D  model  of  an  urban  area  from  multi-view  images 
comprises  the  creation  of  a  Digital  Terrain  Model  (DTM),  representing  the  ground  level,  and  the 
extraction  of  above  ground  structures,  such  as  buildings.  For  cartography  purposes  it  is  also  of 
great  interest  to  identify  roads  and  other  infrastructures. 

Even  though  airborne  laser  scanning  emerges  as  an  interesting  alternative,  photogrammetry 
based  on  aerial  images  remains  a  key  technology  for  3D  cartography  in  urban  areas.  The  new 
generation  of  high-resolution  optical  satellite  sensors  is  a  new  source  of  information  for 
photogrammetric  3D  reconstruction. 

IMST  concentrated  on  photogrammetric  methods  and  report  on  results  obtained  with  commercial 
software  on  the  test  images.  Automatic  building  extraction  from  multi-view  images  is  considered 
as  an  extremely  difficult  task,  mainly  due  to  the  complexity  and  variability  of  the  depicted 
structures.  We  have  had  in-depth  discussions  on  this  topic  with  experts  at  the  Institute  of 
Photogrammetry,  University  of  Bonn,  and  at  the  Ecole  Nationale  Superieure  des 
Telecommunications  and  the  Institut  Geographique  National  in  Paris. 

Commercial  DPWs  are  capable  of  generating  DTMs  of  relatively  good  quality  in  suburban  areas, 
but  in  dense  urban  areas,  extensive  editing  is  necessary  to  derive  the  true  ground  level  and 
eliminate  disturbances  from  buildings.  More  sophisticated  automatic  methods  could  reduce  the 
need  for  operator  intervention. 

Significant  progress  has  been  made  in  the  domain  of  automated  building  extraction  from  stereo 
or  multi-view  aerial  images,  but  automatic  methods  are  not  yet  sufficiently  robust  for  practical 
applications.  However,  semi-automatic  schemes  are  operational,  featuring  efficient  user 
interfaces  and  a  gradual  incorporation  of  stable  automatic  modules. 

More  studies  within  this  filed  will  be  conducted  in  later  phase  of  the  project.  Developments  we 
see  in  services  like  Google  Earth  and  related  projects  will  introduce  solutions  that  the 
Middletown  2010  can  build  on.  We  have  also  initiated  discussions  with  Norway’s  leading 
research  organization  within  this  field  (SINTEF)  that  will  continue  in  2007  and  probably  will 
introduce  solutions  to  these  problems. 

3D  visual  content  and  simulation  production  software: 

Goals 

The  objective  of  this  WP  is  to  study  and  identify  simulation  software  for  the  Middletown  2010. 
This  includes  a  survey  of  technology,  ongoing  projects  and  partners.  The  findings  will  be 
documented  in  a  report.  High  level  requirements  for  simulation  software  will  be  defined. 

Results 

The  project  has  decided  not  to  focus  on  this  work  package  during  phase  I. 

Realistic  and  real-time  medical  3D  simulation  solutions: 


7 


Goals 

The  goal  of  this  WP  was  to  build  an  anatomy  navigator  prototype  that  allows  organ  identification 
(segmentation)  from  human  digital  images  (CT,  MRI)  immerged  with  simulation  of  surgical 
intervention. 

This  includes: 

1)  One  demo  application  that  allows  semi-automatic  and  manually  organ  identification 
(segmentation)  from  human  digital  images  (CT,  MRI)  and  volume  navigation  allowing 
simulation  of  simple  procedures  (e.g.  needle  injections)  -  SimNavigator. 

2)  One  demo  application  of  the  SimSurgery  Education  Platform  (SEP)  where  surgical  simulation 
training  as  instrument  manipulation,  tissue  manipulation,  dissection,  suturing  and  knot-tying  are 
carried  out  by  using  a  low-cost  gaming  platform. 

Results 

The  goal  for  Middletown  is  to  get  a  program  that  enables  us  to  make  3D  objects  from  different 
volumes  so  that  they  can  use  it  as  a  tool  for  creating  digital  organs/objects,  which  can  be  inserted 
into  their  simulation  programs.  This  will  automate  the  process  of  making  realistic  simulation 
objects  used  in  the  Middletown  environment. 

MedVed  is  a  Medical  Voxel  Editor  created  in  co-operation  with  SimSurgery  in  Oslo. 
SimSurgery  works  with  simulations  aimed  at  surgical  operations  and  are  planning  to  develop  a 
simulator  to  use  in  the  course  of  heart-operations.  MedVed  visualizes  anatomic  models  with  use 
of  voxel  graphics  and  to  present  them  as  different  file  structures  on  the  computer. 

MedVed  gives  the  user  an  opportunity  to  make  use  of  algorithms  to  render  a  3D  structure  of  an 
organ.  The  3D  structure  is  displayed  in  three  viewports  that  represent  sagittal,  axial  and  coronal 
views.  Each  of  these  three  viewports  displays  the  structure  in  2D.  A  fourth  viewport  can  also  be 
used.  This  fourth  viewport  displays  the  structure  in  3D. 

The  goal  for  the  project  is  to  create  a  program  that  is  capable  of  viewing  and  editing  data  from 
different  medical  equipment,  for  instance  MRI  and  CT  scanners.  The  program  is  going  to  have 
tools  that  enables  the  user  to  create  objects  from  the  scans  and  save  them  as  objects,  and  some 
measuring  tools  that  enables  the  user  for  instance  to  measure  the  distance  between  two  volumes. 
And  the  program  is  going  to  be  able  to  take  input  from  the  SimPack  (virtual  reality  pins). 

Basic  functions: 

•  Presentation  of  data  in  3  orthogonal  viewports  (axial,  coronal  and  sagittal). 

•  Presentation  of  the  data  in  3D. 

•  Zooming  functions  in  all  4  viewports. 

•  Built-in,  volume  selection  tool  with  visible  edge  detection  line  or  colour  representation  of 
the  chosen  object. 

•  Built-in,  tolerance/quality  adjustment  of  the  volume  fill  algorithm. 

•  Built-in,  editing  on  selected  objects  with  use  of  pencils  (rectangular  pencils). 

•  Continuous  presentation  of  defined  objects  in  the  dynamic  working  area. 

•  Possibility  for  the  user  to  name  objects  and  an  automatic  default  colour  scheme  for  these 
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objects  set  by  the  system. 

•  Image  processing  and  color  conversion  functionalities. 

•  Volume  creation  functionality  using  a  sequence  of  PNG/TIFF  or  DICOM  images. 

Input  formats: 

The  system  supports  the  following  input  formats:  Support  for  8  bit,  12  bit,  16  bit  &  24  bit  colour 
depth  raw  format  files. 

•  Support  for  header  files. 

•  Support  for  8,  12,  16  &  24  bit  PNG  &  TIFF  image  files. 

•  Support  for  8,  12  and  16  bit  uncompressed  DICOM  images. 

And  the  following  output  fonnat 

•  Support  for  storage  in  8  bit  and  24  bit  *.raw  format. 

•  Support  for  header  files. 

•  Support  for  storage  in  8  bit  and  24  bit  PNG/TIFF  format. 


A  view  of  the  systems  main  window  is  shown  below.  This  screen  is  made  with  regards  to  be 
both  well  arranged  and  easy  to  use.  These  are  essential  considerations  under  development  of  a 
good  structure  and  to  meet  the  needs  of  user’s  controllability. 
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Figure  1. 


User  Interface  Overview 


Fast  and  reliable  interaction  with  mobile  devices  to  exchange  and  render  information: 


Goals 

The  goal  of  this  WP  was  to  study  and  describe  how  modem  mobile  devices  and  telecom 
infrastructure  can  be  used  for  interaction  between  the  Middletown  simulator  system  and  real 
people.  The  demo  should  be  based  on  Mobile  Media  Inventions  technology  as  browser,  search, 
blogging  and  other  available  technology  from  the  Mobile  Media  Group  (Video  streaming, 
Gateway  connections  to  operator,  mobile  content  downloading  and  distribution  and  others. 

This  includes: 

1)  One  demo  application  that  illustrates  interaction  between  a  simulated  event  and  a  person 
that  interacts  with  the  ongoing  simulation. 

2)  One  demo  application  which  illustrate  the  use  of  modern  mobile  devices  to  gather  live 
data  from  an  event  and  integrate  this  with  other  available  data  to  be  able  to  fast  build  a 
realistic  view  of  the  situation. 


Results 

The  main  idea  behind  the  prototype  is  to  gather  information  and  knowledge  about  how  modern 
mobile  infrastructure  can  be  utilized  and  how  well  it  can  be  integrated  into  a  simulation  system 
like  Middletown  2010. 

During  the  project  we  have  developed  two  demos  or  prototypes  which  have  been  integrated  to 
one  client-server  application. 

The  prototype  consists  of  a  client  server  system.  The  main  part  is  ’the  client’,  which  is 
implemented  as  a  Java  (MIDP2.0)  application.  Java  MIDP  is  a  technology  that  is  supported  by 
the  majority  of  the  phones  in  the  market.  It  also  represents  the  “less  advanced”  platform,  as  other 
technologies  normally  are  more  “advanced”  and  place  less  constraint  on  the  platfonn.  In  other 
words,  what  works  on  Java/MIDP2.0  should  work  on  other  platforms  as  well. 

The  server  has  not  been  the  main  focus  in  this  prototype  and  only  holds  a  minimum  of 
functionality  necessary  to  serve  the  client. 

The  communication  between  the  client  and  the  server  is  based  on  “internet”  standards 
(GPRS/IP). 
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Server 


The  server  consists  of  3  basics  parts: 


a)  a  control  system  for  emergency  handling. 

b)  an  upload  interface  for  news  and  adverts  with  a  possibility  of  declaring  a  crisis 
situation; 

c)  chat  module  for  the  “Guardian  Angel” 


The  application  can  send  the  GPS  coordinates  to  the  server,  so  we  can  have  an  emergency 

control  system  which  will  improve  the  supervision  and  control. 


There  is  an  interface  on  the  server  from  where  news  and  brandings  can  be  added  or  declare  a 
crisis  situation  that  will  appear  on  the  users  mobile  devices. 


This  is  the  chat  interface  used  by  Guardian  Angel,  who  can  give  medical  or  other  assistance  in  an 
emergency  situation. 

Client 


The  prototype  client  has  been  built  to  be  used  in  two  different  contexts 
and  may  be  more  complex  than  it  will  be  as  a  pure  Middletown 
“simulation  client”  (MC).  The  main  reason  for  this  is  that  we  have  seen  a 
clear  market  need  for  a  system  with  part  of  the  functionality  that  the  client 
represents.  The  need  can  be  seen  as  a  “spin-off’  from  the  Middletown 
system/project,  but  we  have  decided  to  build  one  prototype  as  most  of  the 
functionality  is  overlapping  even  though  it  will  be  used  in  a  different 
context. 


The  client  has  five  main  functions  that  allow  the  user  to: 
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-  contact  help  in  case  of  emergency  (Emergency), 

-  ask  for  help  at  trained  personnel  (Guardian  Angel), 

-  report  his  GPS  position  (Position), 

-  quickly  localize  family  and  friends  (Friends) 

-  submit  information  in  a  crisis  situation.  (Report) 

In  addition  to  the  5  main  functions,  the  client  has  two  related  information  functions.  The 
information  ticker  and  the  “branding  window”. 

Emergency 


The  idea  behind  the  Emergency  function  is  to  enable  the  user  to  get  quick 
and  easy  access  to  help  in  case  of  an  emergency.  This  could  be  the  fire 
department,  ambulance,  police  and  MCC  or  other  important  emergency 
organizations.  The  user  will  be  able  to  quickly  contact  these  emergency 
units  via  a  simple  interface.  The  normal  and  most  useful  way  will  probably 
be  a  voice  call,  but  chat,  sms,  mail  and  a  video 
conference  could  also  be  supported. 


Location 
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The  idea  behind  the  Location  function  is  to  give  the 
user  a  tool  to  help  him  determine  his  own  position  and 
report  this  to  others. 


From  the  location  screen,  the  user  can  ask  for  his/her 
location.  The  prototype  only  supports  GPS,  but  it’s 
also  possible  to  support  location  based  services 
provided  by  the  operators.  The  MC  will  send  the 
device  position  to  the  server  and  receive  a  map  with  the  reported  function  back.  The  map  could 
be  tailored  to  the  situation  and  for  example  contain  graphical  information  about  escape  routes 
etc.  The  prototype  also  keeps  track  of  the  last  reported  positions  (10  in  this  version).  This  could 
help  emergency  personnel  determine  area  where  it’s  most  likely  that  the  user  is  located  in  case 
the  positioning  data  is  not  current. 


Friends 


The  User  will  be  able  to  compile  and  maintain  a  list  of  family  members  and  friends  that  they 
wish  to  stay  in  contact  with  in  an  emergency  situation.  For  each  one,  there  could  be  retrieved 
from  the  server  his/her  current  position  on  a  map.  This  could  be  used  by  both  users  being  in  an 
emergency  situation  and/or  family/friends  looking  for  a  person. 

Guardian  Angel 


The  idea  behind  this  function  is  that  a  person  being  in  any  kind  of 
emergency  situation  can  get  a  “Guardian  Angel”  allocated  to  help  him/her 
out  of  the  situation  or  in  any  other  way  support.  The  prototype  support  both 
chat  and  voice. 


Report 


News  and  branding 


The  idea  with  these  functions  is  to  feed  the  user  with  both  information/news 
and  more  detailed  information  in  case  of  an  emergency. 


The  banner  at  the  top  of  the  client  will  normally  bring  information  to  the  user 
that  is  defined  important  by  the  MCC.  This  could  be  news  about  emergency 
situations  or  more  general  information  related  to  emergency  or  homeland 
security. 

The  “branding  window”  at  the  bottom  of  the  screen  could  normally  be  used  for  more 
information,  adverts  etc,  but  will  in  a  crisis  situation  be  tailored  to  bring  extended,  detailed 
information  to  the  user. 


General  information: 

•  On  the  east  there  Is  an 
explosion.  Take  care.  The 
roads  are  blocked. 

Download  Maps: 

•  map  of  road  51 

•  map  of  east  part  of  town. 


The  idea  behind  the  Report  function  is  to  give  the  user  an  easy  way  to 
report  (to  MCC)  text,  voice  and  pictures.  The  MCC  personnel  can  ask  the 
user  to  take  pictures  (position  is  known)  and  send  them  to  MCC.  The  only 
thing  the  user  needs  to  do  is  click  his  way  through  a  defined 
process/interface. 
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Phase  II  Middletown  2010  Specification: 


Goals 

The  objective  of  this  WP  is  to  describe  the  Middletown2010  simulation  system.  The  result  will 
be  a  requirement  document  to  be  used  as  basis  for  the  detailed  specification  that  will  be 
developed  in  phase  II.  It  will  also  be  used  as  input  to  the  planning  and  budget  process  (re  Plan 
and  Budget  WP). 

Results 


Since  the  goal  of  the  Middletown  2010  ERS,  is  to  develop  a  realistic  interactive  emergency 
simulation  and  response  system,  we  have  found  it  is  far  more  advantageous  to  provide  a  visual 
and  oral  presentation  on  how  the  technologies  interact,  rather  than  in  a  traditional  written 
document.  The  short  demonstration  literally  provides  a  story  or  use  case  of  how  mobile  and  other 
technologies  function  together  to  provide  an  effective  emergency  solution. 


The  story  is  built  around  a  real  event  taking  place  just  outside  Oslo  some  years  back. 


A  realistic  and  recognizable  situation  was  selected  to  demonstrate  the 
practicality  and  usefulness  of  Middletown  2010  and  how  it  can  engage 
professionals,  volunteers  and  the  general  public. 


The  demonstration  also  shows  how  information  is  gathered,  utilized 
and  plotted  visually  to  assist  in  handling  critical  issues  in  an  actual 
emergency. 


Phase  II  Plan  and  Budget: 


Goals 

The  objective  of  this  WP  was  to  develop  a  project  plan  and  budget  for  Middletown  phase  II. 
These  documents  will  serve  as  input  to  the  phase  II  project  funding  process. 
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Results 


Re.  Middletown  2010  -  A  Realistic  Interactive  Emergency  Response  System  for  the  US  Army 

6.  REPORTABLE  OUTCOMES 

Publications:  None 

Presentations:  Middletown  2010-  A  Realistic  Interactive  Emergency  Response  System  for  the 
US  Army,  ATA  meeting,  May  2006,  San  Diego,  CA 

7.  CONCLUSION 

As  stated  above,  the  Middletown  2010  project  needs  to  address  4  fundamental  problems  to 
achieve  the  goal  of  building  the  defined  simulator.  These  are: 

1)  Automatic  generation  of  scenery  models  from  satellite  images  and  photos 

2)  3D  visual  content  and  simulation  production  software 

3)  Realistic  and  real-time  medical  3D  simulation  solutions 

4)  Fast  and  reliable  interaction  with  mobile  devices  to  exchange  and  render  information 

During  the  past  year,  rapid  technological  development  within  all  4  areas  made  Middletown  2010 
more  realistic  than  ever.  Significant  progress  was  made  in  the  field  of  automatic  generation  of 
scenery  models  from  satellite  images  and  photos  and  3D  visual  content.  Progress  in  the 
development  of  simulation  production  software  was  also  achieved. 

Services  like  Google  Earth  and  the  launch  of  more  and  more  advanced  games  and  simulation 
software  for  the  professional  market  (construction,  military,  etc.)  is  a  clear  indication  that  there 
will  be  tools  and  services  available  in  the  market  that  will  help  Middletown  2010  become  a 
reality  at  less  cost  than  earlier  estimated. 

There  still  exist  technological  issues  that  need  to  be  addressed.  One  practical  example  is  how  to 
create  high  quality  models  of  urban  places  (houses,  buildings,  etc.)  based  on  air-  and  satellite 
photos.  We  have  found  projects  within  the  Norwegian  research  organizations  (NR  and  Sintef) 
that  address  this  problem  but  they  lack  sufficient  funding  to  produce  results  and  solutions.  The 
ideas  and  knowledge  are  there  but  the  researchers  lack  resources. 

Middletown  2010  also  needs  to  address  the  integration  issue  more  in  depth.  This  has  been 
addressed  from  the  start  but  the  development  towards  more  open  and  standard  solutions  for  data 
and  service  interaction  suggests  resources  can  be  saved  by  integrating  Middletown  with  existing 
solutions.  Our  plan  is  to  build  upon  what  is  already  invested  in  other  projects. 

To  be  able  to  move  forward  in  this  field,  we  need  to  work  more  closely  with  existing 
organizations  and  systems  that  can  provide  what  Middletown  requires.  Therefore  we  have 
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decided  to  put  some  extra  resources  towards  the  development  of  a  multimedia  demo  and  not 
develop  a  traditional  textual  specification.  We  hope  to  reach  decision  makers  who  normally  do 
not  have  time  to  read  technical  specs  and  show  how  Middletown  can  fit  into  the  total  picture. 

The  work  on  the  two  other  fundamental  issues  has  developed  according  to  plan.  We  have  built 
demos  that  prove  our  ideas  and  we  have  even  identified  “spin  offs”  like  the  mobile  Middletown 
client  that  offer  solutions  faster  than  expected.  The  solutions  can  serve  as  both  a  stand  alone  and 
integrated  Middletown  module. 

In  the  next  phase,  we  will  continue  to  address  some  of  the  fundamental  problems  that  already  are 
addressed  in  this  phase.  We  will  focus  more  on  the  user  interface  (how  the  user  interacts  with  the 
Middletown  system).  We  will  continue  the  development  of  the  mobile  application  and  try  to  find 
solutions  that  support  the  research  of  building  urban  3D  models  faster  and  with  higher  quality. 

8.  REFERENCES 

None 
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